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(Substitute Specification) 



Continuation-in-Part Application 
A Video Camera Utilizing Sequential Diversity Imaging For Image Clarification 

Background of the Invention 

The present invention is a Continuation-in-Part of Application No. 09/905.693 
eMtjed i j:Seguejitlaj..Djyersit^ 

to imaging through a random optical medium, such as the turbulent atmosphere 
or the slowly changing characteristics of a video camera. 

Wavefront sensing is employed to determine the distorting wavefront caused by 
the turbulent atmosphere and one or more the following methods are generally 
employed to provide an estimate of the distorting wavefront. . 

1 . Dithering. This method of wavefront sensing continuously changes the 
adaptive optics and monitors the image quality of the observed image. US Patent 
3,979,585 entitled "Adaptive imaging telescope with camera-computer transform 
image quality sensing and electro-optic phase shifting " and the publication by M. 
A. Voronstov and V. P. Sivokon, entitled "Stochastic parallel-radiant-descent 
technique for high-resolution wave-front phase-distortion correction," J. 
Opt. Soc.Am.,A,1 5,2745 (1988) are examples of this approach. They are 
sequential techniques, in the sense that a physical property of the adaptive optics 
is changed, and if the next (time-sequential) image is sharper the change is 
increased, otherwise the change is reversed. This is a physical search for "best 
focus" and is how the human eye and most cameras work. 
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2. Shearing Interferometer. This method of wavefront sensing uses a 
reference beam to create an interference pattern, from which the unknown 
wavefront is estimated. It requires a laser-based interferometer. US patent 5- 
3,923,400 entitled "Real-time wavefront correction system film a nim a tion 
technique" describes this approach. 

3. Shack-Hartmann Sensor. This device employs an array of lenses to focus 
multiple, small images, each seen through a different section of the aperture, onto 
a detector. Shifts in the small images are caused by local tilts in the waveform, 
which allows the wavefront to be reconstructed. US Patents 4,141 ,652 entitled 
"Sensor system for detecting wavefront distortion in a return beam of light" and 
5,350,91 1 entitled "Wavefront error estimation derived from observation of 
arbitrary unknown extended scenes" disclose the use a Shack-Hartmann sensor. 

4. Curvature Sensing. This method is described in the publication by F. 
Roddier et al._entitled "A simple low-order adaptive optics system for near-infrared 
applications," Publications of the Astronomical Society of the Pacific, 103, 131 

(1 991 ), whereby two or more images are measured along the path of the optical 
system. The local curvature of the propagating wave is determined and it is 
propagated, by computer calculation, back to the aperture to form the wavefront 
estimate. 

5. Phase diversity. This method employs diverse images, measured 
simultaneously whereby the diversity is a quadratic phase shift, which can be 
introduced by defocusing the optical system. Additional equipment in needed to 
record the out-of-focus image. US Patents 4,309,602_7entitled "Wavefront 
sensing by phase retrieval", 5,384,455 entitled "Measurement-diverse speckle 
imaging", 5,610,707 entitled "Wavefront sensor for a staring imager", 6,107,617 
entitled j'-Liquid crystal active optics correction for large space based optical 
systems ", and the publication by R. Paxman, et al. T entitled "Optical 
misalignment sensing and image reconstruction using phase diversity," J. Opt. 
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Soc. Am., A, 5, 914 (1988)^ each use phase diversity to estimate the wavefront. 
It is noted that the phase diversity approach to wavefront sensing was used to 
determine the aberration in the Hubble telescope project. 

Eaeh-ef-#H3-ajefeffl ef^ 

Note that all of the patents mentioned in the section on phase d iversity use an jry 
focusJmam^^ 

sensing, as op posed to the current invention which uses se quential, in-focus 
jmjagesmdmgu^ 

adaptive optic device. The invention does not need additional equipment . such 
as an out-of-focus image, and funct i ons to control an adaptive optic device 
■ae-apfrve--efft-tes in the camera system. 

One purpose of the instant invention is to provide a system for determining 
aberrations within a video camera or caused by the optical medium and to 
eliminate the aberrations using no additional optics or sensors . e x i st i n g ca mem- 

Summary of the Invention 

The invention describes a video camera which produces digital cla rified images of 
an object which is distorted by time-vary in g aberrations in the optical medium, it 
uses an adaptive optics device to eliminate the aberrations and a sequential 
diversity processor to produce the control signals for the adaptive optic device. 
T±e se^e^ 

camera, u ti tizes - ffl - tar - aetive-s^^ 

determini ng th o ab e rrat i ons caused by the m e d i um and contro l s o n a d a pti ve-epfe- 
ifh#>e--8fft4e^ 

■imaging--us€S--eee--er--fflere-ef-4h o i ma g os , each with a known diversity such as 
phase, wavelength, or spatial shift, to deduce both the unknown object and the 

aberrations in parameters of the jjpiicalmedium. Sequential frames are the 
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diverse images and When images of the object aro r ecorded os a video sequence 
aR#-wj^n--&R"a#af^ the changes 

in the adaptive optics, per se, are used as the diversities. The sequential frames 
comprise the diver s i ty i mages and o standard seque ntial p r oc es sor is employed 
te-eerrtre l the a d a pti ve opt i c. Unlike other diversity imaging methods no other 

needed to control the adaptive optic device. 

Brief Description of the Drawings 

Figure 1 is a diagrammatic representation of the imaging system, which employs 
an adaptive optic and a sequential diversity processor within a video camera, in 
accordance with the invention; 

Figure 2 is a flow chart representation of the imaging system and the sequential 
diversity algorithm used within the sequential diversity processor of Figure 1 ; and 
Figure 3 depicts a computer simulation of 6 frames of a video sequence achieved 
within the video camera of Figure 1 . 

Description of the Preferred Embodiment 

The sequential diversity imaging system 12 within the video camera 13 depicted 
in phantom in Figurel, receives an optical signal 1 which may have aberrations 
introduced by a randomly changing opticas medium. The optical signal passes 
through the camera aperture 2 and is imaged by a lens 3. Between lens 3 and 
detector 5 is an Adaptive Optic 4, hereinafter, "AO". - This portion of the 
ipveMon^ as described in the publication 

entitled "Phase retrieval and diversity in adaptive optics," R.A. Gonsalves Opt. 
Eng., 21, 829 (1982) and as further described in US Patent r4.309.6Q2. The 
optical system produces a clarified in-focus optical image of the distorted object In 
the focal plane of the lens 3. The AO 4 modifies the optical properties of the 
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optical energ y it receives from the lens so as to can cel aberrations in the object 
being viewed, it also receives digital signals from the sequential diversity 
processor 7. signals which control the AO 4. 

The detector 5 is located in the focal plane of the lens and is an array of 
photodiodes, not shown, each of which provides a signal of magnitude related to 
the intensity of radiant energy incident thereon. The AO is controlled as described 
herein, to reduce the wavefront error signaL. as-desef i bed- ^tffr-th-e- 
■aferemef rt i oncd US patent 4,300,602. The output of detector 5 is a video 
sequence of digital images, as indicated at6.__This sequence of digital in -focus 
images is the input to a Sequential Diversity Processor 7 which produces digital 
control signals 8 to control the configuration of the AO, so as to cancel the 
aberrations introduced by the random optical medium. The AO could comprise a 
high-resolution device to allow a wide range of correction mechanisms, such as 
Zernike polynomial fitting of a complicated wavefront , or tho automatic focus i ng 
and reg i otrat i on mechan i sms used -i n ■ eonsumer camcefd e fs . - 

Figure 2 depicts a f l ow d i agram dep i ct i ng the image flow through the lens 3, AO 4, 
detector 5 and the processing of data within the Divers ity ..A|gorjthm.10..ajid.the.. 
Predicted Algorithm 11. the Sequent i al Divers i ty Processor 7 as shown i n F i gure 
4-r The k th frame of the video output 6, l(k), is an input to the Diversity Algorithm 
10. Other inputs to the Diversity Algorithm 10 are successively delayed versions 
of l(k), namely l(k-1), l(k-2), which are delayed and stored in a digital buffer 
Delays 9A . w i^n-tRe-SeqtteRtM-B i ve^ 
DeleyS"9A-OG-as-defiicte^-in-Figufe-2v 

The k th output of the Diversity Algorithm, D(k), and delayed versions from Delays 
9B t-hereef, are used as inputs to the Diversity Algorithm 10. D(k) is an input to 
the Predicted Algorithm 1 1 which predicts the change in the random optical 
medium for the next frame of the video sequence and controls the AO 4 with a 
control signal T(k) as indicated at 8. The other inputs to the Predicted Algorithm 
1 1 are delayed versions T(k-1 ), T(k-2) .— . from Delays 9C. of the control signal 
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T(k-2) of tho control signal T(k). 

To further describe the flow diagram of Figure 2, the following notations assume 
that the media aberration is due to an unknown wavefront distortion, which is 
typical of atmospheric distortioni „" The €a l €u l at i eRS - w^ 
10 of F igur e 2 e m p l oy the fo ll ow i ng:. 

W(k)_= Unknown distorting wavefront at time k. 

T(k) = Phase on the AO at time k. 

C(k) = Compensate Residual phase to be estimated 
I =W(k)+T(k). (1) 

l(k) = Measured image at time k. 

D(k) = Diversity phase. 
With l(k-1 ) as the first image and l(k) as the diversity image, the diversity phase 
D(kX.©k is the change in the AO phase from time k-1 to time k, such that: 
| D(k) = T(k) - T(k-1 ) . (2) 

Let Q( k ) represent the estimate o f C ( k), as in e q uation (1), 

T-e-es- t i matc the res i dua l phase, \ ci C(k) represent the est i mate Q(k) from e qtefeR- 

| Q(k) = W1(k) + T(k), (3) 

where W1(k) is an estimate of W(k), the unknown phase at time k. SH^€ - e-T-( - k-) - 4s- 
known, at t i me k * 1 the AO phase i s At time k + 1 the AO should be set to the 
negative of the unknown distorting wavefront W (k+1) at time k+1 . An estimate of 
the distorting wavefront at time k, W1(k), provides a good estimate of W(k+1) 
when AO updates are within the time constant of the changing optical medium. 
Therefore: 

T(k+1)-= -W1(k), (4) 

which tends to cancel the wavefront distortion at k+1 . 

Solving (3) for W1(k) and substituting into equation (4), results in 
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-T(k+1)~= - Q(k) +T(k) ,which implies 
_T(k) „=-Q(k-1) +T(k-1) . 



(5) 



£ef To find the diversity phase D{k) Qk insert equation (5) into equation (2) 
whereby 

D(k)_= ( - Q(k-1 )+ T(k-1 ))- T(k-1 )= - Q(k-1 ) . 
The new diversity D(k+4) is now defined as 

| D(k+4) = -Q(k-Vfr (6) 

Wwhich provides the diversity at time fc*4. 

Inserting equation (6) into equation (5) results in &p the new AO control 
sjaoaLpheser 

| T (k+1) = T ( k) + D( k + 1). F) 

I T(k) = D(k) + T(k-1). (7} 

whim, is, a simple restatement of equation (2), 

Equations (6) and (7) result in the i nteract i ve sequential diversity imaging 
function in accordance with the invention. 

In summary, the sequential diversity algorithm 10 and predicted algorithm 11 of 
Figure 2 contained within the sequential diversity processor 7 of Figure 1 employ 
the diversity D(kJ_) and the diverse images, l(k) and l(k-1), to calculate an 
estimate Q(k-1) of the residual phaseT QC(k£L) J _,-w h i € - h --i s - 4he - ®uffl -- &f --t he- 
weve fr eRt- <j f stert4&n"p j uS"the AO setting T T(k>: - - The new diversity, D(k«*4), is set to 
the negative of Q(k£l), according to equation (6), and it is added to the euffen^AQ- 
phes&4 e - fefffl - #ie"neW"AQ"ph - ase" T{k '» 1");" previous AO setting to form the new 
AO^eJtingJiJ^Laccording to equation (7). 

A-s-eRown4R--Rg-.-2, the inpu ts to tho d i vers i ty a l g orithm 1 0 ■afe--the--measu-r-ed" 
dtvefsit-ie9-0(k-4-);--0(-k-2-)-vta- dolays OB. The inputs to the p redicted algof-ithm-44- 
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arc the estimated phooo di v ersi ty Dk; and the phasco T(k 1), T(k -2) via delays 

Q&r- 

Figure 3 depicts a computer simulation of frames 1-6 as achieved within the 
sequential diversity processor 7 within the video camera 13 of Figure 1 . The 
object 17 comprises four lines of text viewed through a turbulent atmosphere 1 
via the video camera lens 3. The original unprocessed images 15 of the text are 
shown in lower regions of each frame and the processed images 16 are shown in 
the upper regions thereof. Comparing the original images 15 to the processed 
images 16 within each frame depicts how the processed images are clear and 
readable within 6 frames. The Strehl ratio, the ratio of the peak value of the 
unprocessed Point Spread Function, "PSF" to the peak value of the processed 
PSF as described in the aforementioned publication "Phase retrieval and diversity 
in adaptive optics," wherein higher Strehl ratio defines higher image quality is 
now employed to determine image quality for the original unprocessed images 15 
and the processed images 16. For the 6 original unprocessed images depicted 
in Fig. 3, the average Strehl ratio is 0.09 whereas the average Strehl ratio for the 
processed images is 0.48, which results in an improvement of about a factor of 5. 

Accordingly, the invention teaches i nteract i ve sequential diversity imaging within 
a video camera for determining the aberrations caused by the optical medium 
and for controlling an adaptive optic device contro l s an adopt i ve opt i c in the 
optical system to eliminate the aberrations. The adaptive optic device clarifies the 
image as it appears on the detector in the lens' focal plane and the detector 
produces a sequence of digital clarified in -focus images. These solely in-focus 
images are the input to the sequential diversity processor which estimates the 
resjoLuatpM 

the diversity algorithm, and the control signal for the AO device. No out-of focus 
images are needed nor is an additional detector needed to measure a defocus 
image, as is the case with conventionai phase diversity systems identified in the 
above section, entitled Background of the invention. 

8 



